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System and Method for Intelligent 
Buffering and Bandwidth Allocation 

BACKGROUND 

5 Field of the Invention 

This invention relates generally to the field of multimedia systems. More 
particularly, the invention relates to a multimedia system for storing and 
coordinating the distribution of multimedia content and data over local and 
remote communication channels. 

10 Description of the Related Art 

in 

ffi Thirty years ago, only a limited number of multimedia devices were 

^ available to consumers. These included, for example, radio receivers for playing 

!, I AM/FM radio, television sets for displaying broadcast video programs, 

13 turntables for playing vinyl records, and standard telephones for communicating 

= fi 

Q 15 across long distances. These devices were relatively easy to use and were the 
;*? only devices capable of playing the media for which they were designed (e.g., a 

standard LP record could only be played back on a turntable). 

Since that time, the digital revolution has produced a seemingly unlimited 
number of new multimedia devices. For example, audio and video content 
20 today may be digitally encoded on compact disks ("CDs"), digital video disks 
("DVDs"), MiniDisks™, digital audio/video tapes, and VHS tapes. Moreover, 
with the proliferation of high speed Internet access through digital subscriber 
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lines ("DSL") and digital cable systems, consumers are frequently turning to the 
Internet to retrieve, store and play back audio and video content (e.g., in "MP3" 
or "MPEG-2" format, respectively). 

In addition, the number of multimedia channels over which consumers 
receive audio, video, and data has increased significantly. Today these include 
standard broadcast television; digital /analog cable television; various direct-to- 
home satellite broadcast systems (e.g., DirecTV™, the Dish Network™); Internet 
service via dial-up, DSL and cable; AM/FM radio; and standard telephone 
service. 

As a result, consumers are burdened with coordinating a variety of 
incompatible multimedia types and multimedia communication channels. This 
is not merely burdensome, but also costly and inefficient in that consumers are 
required to purchase a variety of different stand-alone decoder /playback and 
encoder/recorder devices, and subscribe to a plurality of incompatible 
multimedia services (e.g., standard telephone service, digital cable service, DSL 
Internet service, . . . etc). 

In addition to the foregoing problems experienced by consumers, serious 
problems have arisen in recent years for artists and other content creators. More 
specifically, copyright infringement has become rampant due to the fact that 
digital audio and video can easily be copied from one media type to another 
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without any noticeable loss in signal quality (e.g., from the Internet to a 
computer hard drive to a recordable compact disk). 

Current device integration and home networking solutions such as those 
designed by the Universal Plug and Play Forum ("UPNP"), and Jini™ provide 

5 some level of interconnectivity between digital devices in the home. However, 
these solutions focus on connecting normally autonomous, stand-alone digital 
devices on a peer-to-peer network and, as such, often result in no cost savings to 
the end user. For example, the devices themselves are relatively expensive 
because they must be "smart" enough to function outside of the network 

10 environment and known network resources. In addition, these solutions can be 
extremely difficult to install and configure due to the fact that each device must 
be equipped with a proprietary interface for communicating to other devices on 
the network. 

Accordingly, what is needed is a centralized, comprehensive, open system 
15 capable of storing and coordinating between a variety of different media types 
and media transmission channels. What is also needed is a system and method 
which will protect copyright holders' interests in the underlying audio /video 
material. What is also needed is a system and method which will work 
seamlessly with both currently available standalone devices (e.g., personal 
20 computers, televisions) as well as non-standalone devices (e.g., Web terminals 
and network telephones). 
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SUMMARY OF THE INVENTION 

A method is disclosed comprising: identifying histogram data associated 
with multimedia content; and transmitting the multimedia content at a specified 
bitrate based on the histogram data. 
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BRIEF DESCRIPTION OF THE DRAWINGS 



A better understanding of the present invention can be obtained from the 
following detailed description in conjunction with the following drawings, in 
which: 

FIG. 1 illustrates a network architecture for implementing embodiments 
of the invention. 

FIG. 2a illustrates one embodiment of a home media server hardware 
architecture. 

FIG. 2b illustrates one embodiment of a home media server software 
architecture. 

FIG. 3 illustrates a plurality of media modules installed in one 
embodiment of a home media server. 

FIG. 4 illustrates one embodiment of a home media server which includes 
a DVD/CD/CD-RW drive. 

FIG. 5 illustrates a home media server communicating to two 
audio /video media nodes over a wireless network. 

FIG. 6a illustrates one embodiment of a multimedia node hardware 
architecture. 

FIG. 6b illustrates one embodiment of a multimedia node software 
architecture. 



TCW 



6 



04259.P008 




FIG. 6c illustrates one embodiment of the system for tuning between 
multimedia channels. 

FIG. 6d illustrates one embodiment of the system for playing a CD 
jukebox. 

5 FIG. 7a illustrates one embodiment of a home media server for 

coordinating between standard telephone services and IP telephone services. 

FIG. 7b illustrates one embodiment of a software architecture for 
implementing standard telephone, IP telephone and /or video conferencing. 

FIG. 7c illustrates a signal diagram of communication between a home 
10 media server and telephone. 

FIG. 8a illustrates a home media server coordinating between standard 
broadcast channels and packet-switched channels (e.g., the Internet). 

FIG. 8b illustrates a potential progression of bandwidth allocation 
between packet switched channels and analog/ digital broadcast channels. 

15 FIG. 8c illustrates multimedia buffering according to one embodiment of 

the invention. 

FIG. 9a illustrates a histogram showing a normalized bitrate for a 
particular multimedia stream. 

FIG. 9b illustrates one embodiment of a system for intelligent bandwidth 
20 allocation and buffering. 
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FIG. 9c illustrates bitrate data normalized at one second intervals. 

FIG. 9d illustrates bitrate data for three separate multimedia streams 
normalized at 10 second intervals. 

FIGS. 9e-m illustrate histograms of normalized bitrate data for various 

DVDs. 

FIG. 10 illustrates a cable television module according to one embodiment 
of the invention. 

FIG. 11 illustrates a satellite module according to one embodiment of the 
invention. 

FIG. 12 illustrates a cable modem module according to one embodiment 
of the invention. 

FIG. 13 illustrates copyright tags implemented in one embodiment of the 
invention. 
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DETAILED DESCRIPTION 

In the following description, for the purposes of explanation, numerous 
specific details are set forth in order to provide a thorough understanding of the 
present invention. It will be apparent, however, to one skilled in the art that the 
invention may be practiced without some of these specific details. In other 
instances, well-known structures and devices are shown in block diagram form 
to avoid obscuring the underlying principles of the invention. 

Embodiments of a Centralized Multimedia 
and Computing System 

System Overview 

As illustrated in Figure 1, in one embodiment of the invention, a digital 
media server 110 equipped with a processor and a mass storage device acts as a 
central repository for decoding, storing and distributing multimedia content and 
data. More particularly, the digital media server 100 coordinates multimedia 
content from Internet communication channels 120 (e.g., DSL, cable Internet), 
broadcast communication channels 130 (e.g., digital/ analog cable, satellite), 
and/or Public Switched Telephone Network ("PSTN' 7 ) communication channels 
170 (i.e., standard telephone) to provide a stable, real-time home media network 
140 for a plurality of network devices 150-151, 160-166. 
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One Embodiment of a Home Media Server Architecture 
As illustrated in Figure 2a, one embodiment of a home media server 110 
computing architecture includes a central processing unit 200 capable of 
processing data and multimedia content stored in main memory 201 and a mass 

5 storage device 230 for storing data and multimedia content. In one embodiment, 
the central processing unit 200 is a Pentium®-class processor such as a Pentium 
HI® operating at a 1 GHz or faster clock frequency. It should be noted, however, 
that the underlying principles of the invention are not limited to any particular 
processor speed or processor type. The main memory 201 may be a random 

10 access memory or any other dynamic storage medium (e.g., SDRAM, DDRAM, 
RD-RAM, . . . etc). The mass storage device 230 of one embodiment is capable of 
storing hundreds, or even thousands of hours of multimedia content (e.g., 
movies, digital audio, . . . etc) as well as other types of digital data (e.g., computer 
programs, word processing documents, . . . etc). Devices transmit and receive 

15 data to /from the mass storage device 230 over a high speed interface such as an 
enhanced IDE interface with Ultra DMA capabilities or a Small Computer 
System Interface ("SCSI"). However, various other interfaces may be employed 
while still complying with the underlying principles of the invention. 

An application-specific integrated circuit ("ASIC") 210 coordinates 
20 communication between the various system components and offloads certain 
designated processing tasks from the CPU. The ASIC may be custom built based 
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on the requirements of the home media server 110 or may be built using gate 
arrays, standard cells or programmable logic devices. 

Communication modules 240-245 electrically coupled to the home media 
server 110 via a system bus 220, allow the home media server 110 to 
communicate over different local and remote communication channels. In one 
embodiment, the system bus 220 is a peripheral component interconnect ('TCI") 
bus, although various other bus types may be configured within the home media 
server 110 (e.g., ISA, EISA, Micro Channel, VL-bus . . . etc). 

In the particular embodiment illustrated in Figure 2a, the communication 
modules 240-245 electrically coupled to the system bus 220 include an RF 
network module 240 for communicating over the home media network 140 (i.e., 
via a wireless RF channel), a cable TV module 241 for receiving broadcast cable 
channels, a cable modem module 242 for providing Internet access via a cable 
system (i.e., using the TCP/IP protocol), a satellite TV module 243 for receiving 
satellite broadcasts, and a DSL module 244 for DSL Internet access. Moreover, a 
virtually unlimited number of new modules may be added as necessary to 
support new or existing communication channels/ protocols (as indicated by 
module 245). 

Other components within the home media server 110 architecture include 
an MPEG-2 decode module 202 (and /or other decode modules such as AC3, 
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MPEG-1, . . . etc); an audio module 203 comprised of a digital-to-analog 
converter, a Sony-Philips Digital Interconnect Format ("SP-DIF") interface and a 
standard telephony interface for providing digital and analog audio and 
standard telephone service to external audio /telephony devices; an Ethernet 
5 port provided directly the system ASIC 210 (as indicated by the "100 Base-T 
Ethernet" designation); a Firewire (IEEE 1394) port 204; a Universal Serial Bus 
("USB") port 205; and an infrared port 206. Various other communication 
interfaces may be configured in the system, either directly on the primary home 
media server architecture 110 (e.g., on the media server 110 "motherboard"), or 
hIJ 10 as an add-on module 240-245. Moreover, the communication modules (e.g., 202- 
1/1 206), the CPU 200 and/or the memory 201 may be incorporated within the 

system ASIC 210, rather than as separate modules as illustrated in Figure 2a. 

Q Embodiments of the home media server 110 may also be equipped with a 

j=f DVD drive, CD player, CD Read-Write drive, recordable DVD drive (as 

S 15 described in greater detail below), and/or any other type of portable storage 
medium 235. In one embodiment, these devices may communicate with the 
home media server 110 via an AT Attachment Packet Interface ("AT API"), 
although the type of interface used is not pertinent to the underlying principles 
of the invention. 

20 Figure 2b illustrates a software architecture employed in one embodiment 

of the home media server 110. Different hardware architectures 280 may be used 
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to support the software, including the hardware architecture illustrated in Figure 
2a. A multitasking, multithreaded operating system ("OS") 270 (e.g., Linux, 
UNIX, Windows NT®) with real time streaming support is executed on the 
hardware 280. In one embodiment, certain proprietary customizations 292 are 
programmed in the media server OS 270 including, for example, real time 
services for streaming audio and video (real time OSs typically do not include 
these features). 

A media toolkit 260 executed within the home media server 110 provides 
an application programming interface ("API") for the different media server 110 
applications described herein as well as a set of media server 1 10 utilities. In one 
embodiment, a minimum quality of service ("QoS") is defined within the media 
toolkit 260. The media toolkit 110 may be comprised of several functional layers 
including a media stream abstraction layer; a media stream resource 
management layer; a security/ conditional access layer; and a transport/ storage 
abstraction layer. 

In one embodiment, the home media server 110 is configured with 
support for the realtime transport protocol ("RTP") and the realtime streaming 
protocol ("RTSP") (see, e.g., RTP/RTSP module 251). Briefly, RTP is an IP 
protocol which supports real time transmission of voice and video. An RTP 
packet typically resides on top of a user datagram protocol ("UDP") and includes 
timestamping and synchronization information in its header for proper 
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reassembly at the receiving end. RTSP is a well known protocol for streaming 
multimedia content over a network. It should be noted, however, that various 
other streaming protocols may be implemented while still complying with the 
underlying principles of the invention (e.g., Quicktime®, Windows Media, . . . 
etc). 

RTP and RTSP were designed primarily for PC to PC communication over 
non-realtime networks. Accordingly, because one embodiment of the home 
media server 110 operates over a realtime home media network 140 and 
communicates to devices other than PCs (e.g., media nodes 150, 151), certain 
optimizations 290 may be programmed within the RTP/RTSP component 251. 
These include, for example, support for multiple data streams between the home 
media server 110 and the various multimedia devices 150, 151; 160-166 (e.g., one 
or more video streams and several audio streams) 

Apache HTTP server support 250 is also included in one embodiment of 
the home media server 110. Apache is an open-source HTTP server software 
product which may be executed on various modern desktop and server 
operating systems. Once again, certain media server customizations 290 may be 
included within the Apache component 250, further refining HTTP support for 
the home media network 140 environment. 
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In one embodiment, a plurality of device servers 252 are executed on the 
home media server 110. These devices servers provide application support for 
each of the media nodes 150-151 and other devices 160-166 communicating with 
the home media server 110 over the home media network 140. For example, in 
5 response to a "tune" command sent from a media node 150, a tuning device 
server will cause the home media server to tune to a broadcast channel specified 
by the tune command. The tuning server may also include conditional access 
functionality (i.e., only allowing tuning to occur for channels to which the user 
has access rights). 

■■aw 

?5 10 Although described above as a "software" architecture, it should be noted 

U! 

JTj that various elements illustrated in Figure 2b may also be implemented in 

firmware and/or hardware while still complying the underlying principles of the 
O invention. 

q Referring again to Figure 1, numerous digital and analog devices may be 

p 

15 configured to communicate with the home media server 110 over the home 
media network 140. By way of example, and not limitation, these include 
personal computers 160, cameras or digital camcorders 161, printers 162, 
notebook computers 163, automotive audio/video systems 164, cell phones or 
personal digital assistants 165, standard telephones 165 (including fax machines), 

20 home security systems (not shown); and /or home climate control systems (not 
shown). 
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In one embodiment, complex multimedia and data processing such as 
tuning to selected channels, recording of specified programs/music, storing 
phone numbers and personal data, connecting to remote network sites, etc., is 
performed at the media server 110, rather than at the individual network devices 

5 160-166. As such, these devices 160-166 may be manufactured relatively 

inexpensively. For example, multimedia nodes 150, 151 may be equipped with 
just enough processing power and memory to receive and play back a 
multimedia signal, with storage and control (e.g., tuning) functions offloaded to 
the home media server 110. Similarly, a telephone 166 may be designed with 

10 nothing more than a low power microcontroller coupled to an RF transmitter, 
with telephony functions and contact data storage provided by the home media 
server 110. In addition, because these network devices 160-166 do not require as 
much circuitry, they will be lighter than currently-existing devices, and will 
consume less power. 

15 In one embodiment, the primary communication medium over which the 

home media server 110 and the various devices 150-151, 160-166 communicate is 
wireless RF (e.g., via network module 240), with terrestrial transport connections 
such as Ethernet reserved for devices which are not within RF transmission 
range. Moreover, certain devices which require a substantial amount of home 

20 media network 140 bandwidth (e.g., high definition television 170), and/or 

devices which are in close proximity to the media server 110 may be configured 
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to communicate over terrestrial transports, depending on the requirements of the 
particular configuration. 

One Embodiment of a Multimedia Node Architecture 
Distributed multimedia nodes 150 and 151 illustrated in Figure 1 provide 
5 an interface to the home media network 140 for audio systems 170 (e.g., audio 
amplifiers and speakers) and/or video systems 171 (e.g., standard television sets, 
wide screen television sets, high definition television ("HDTV") sets, or any other 
device capable of displaying video). 

As illustrated in Figure 6, one particular embodiment of a multimedia 
10 node architecture (see, e.g., node 150 in Figure 1), is comprised of a network 
interface 605 coupled to the multimedia node bus 610 (e.g., a PCI bus); a system 
ASIC 620, including MPEG-2 decode /graphics logic 630 (or other multimedia 
decode logic); a central processing unit 640 and memory 650; an audio processing 
unit 660; and /or an infrared port 670. 

15 The local central processing unit 640 and memory 650 execute programs 

and process data locally (i.e., at the multimedia nodes 150, 151). The network 
interface 605, which may be an RF interface or a terrestrial interface (e.g., 
Ethernet), receives/transmits multimedia content and control data from/ to the 
home media server 110 over the home media network 140. The system ASIC 620 

20 decodes and processes incoming multimedia content transmitted from the home 
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media server 110 via MPEG-2 decode/ graphics logic 630 (or other multimedia 
compression standard) to produce one or more video outputs 680 (e.g., an NTSC 
output, an HDTV output, . . . etc). In one embodiment, a separate audio 
processing unit 660 produces both digital and analog audio outputs, 681 and 682, 
respectively. Moreover, one embodiment also includes a local mass storage 
device (not shown) for storing certain multimedia content and /or data (e.g., 
frequently-requested content / data) . 

In one embodiment, the local infrared interface 670 receives control 
commands from a remote control unit (e.g., unit 532 in Figure 5) operated by a 
user. As described below with reference to Figure 5, control data/commands 
received through the infrared interface 670 may subsequently be transmitted to 
the home media server 110 for processing. For example, in one embodiment, if a 
user selects a command to change to a live broadcast channel (e.g., the evening 
news), the command is transmitted from multimedia node 151 to a home media 
server 110 timing application, which forwards the command to a video module 
(e.g., CATV module 241, satellite TV module 243). The tuner in the video 
module (see, e.g., timer 910 in Figure 9) then tunes to the requested frequency 
and the new video signal is streamed from the home media server 110 to the 
multimedia node 151. Alternatively, or in addition, if the requested content was 
previously stored on the home media server 110 (e.g., on mass storage device 
230), then no broadcast tuning is necessary - the content is simply read from the 
storage device 230 and transmitted to the multimedia node 150. Accordingly, as 
TCW 18 04259.P008 



described in greater detail below, the amount of audio /video content which 
must be broadcast to the home media server 110 (i.e., over dedicated broadcast 
channels) is significantly reduced. 

One embodiment of a multimedia node 150 software architecture is 
illustrated in Figure 6b. Various hardware architectures 690 may be used to 
support the software architecture, including the architecture illustrated in Figure 
6a. An operating system 691 executed on hardware 690 (e.g., Windows '98, 
Linux, . . . etc) includes client customizations for optimizing communication over 
the home media network 140 (e.g., providing low level support for real time 
streaming of audio and video). 

A standard set of user interface components 694 included in one 
embodiment may be employed (e.g., by application developers) to generate 
unique interactive interfaces at each of the media nodes 150, 151. For example, a 
user-navigable tuning index may be included which lists available content by 
dates/times and allows users to graphically select a particular broadcast channel 
and/or stored content from the mass storage device 230. 

In addition, support for the hypertext markup language ("HTML") 
and/or the extensible markup language ("XML") are included in one multimedia 
node 150 embodiment, allowing users to download, view and modify various 
types of Internet content (e.g., Web pages). 
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Applications executed on multimedia nodes 150, 151 may include client- 
specific applications 697 and/or home media server 110 control applications 696. 
Two examples of media server control application 696 will now be described 
with reference to Figures 6c-6d. It should be noted, however, that these 
5 examples are for the purpose of illustration only and should not be read to limit 
the scope of the invention. 

Figure 6c illustrates an embodiment for tuning to a particular broadcast 
channel (e.g., cable, satellite broadcast). A user initially selects a particular 
channel (e.g., via a remote control device) to be viewed at the local multimedia 
10 node 151. The command is received by the local TV tuning application 601 

executed on the multimedia node 151, which sends a tuning request to the home 
media server 110 over the home media network 140. A timer server module 602 
executed on the home media server 110 receives the request and forwards the 
request to the timer driver 271 (e.g., through the media toolkit 260). In response 
15 to the command, the tuner driver 271 directs the tuner hardware (see, e.g., tuner 
910 in Figure 9 described below) to tune to the specified channel. The tuner 
driver also communicates with the conditional access subsystem (see, e.g., 
module 930 in Figure 9) to determine whether the user has the right to view the 
particular channel (e.g., the channel may be a subscription-based channel such as 
20 HBO). 
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If the user has access rights, then the tuner server module 252 creates 
video stream for the specified broadcast channel and feeds the stream to the 
multimedia node 151. In one embodiment, the tuner server module 602 creates 
the stream using the RTP/RTSP protocol (i.e., by communicating with the 
5 RTP/RTSP module 251). The media node receives the stream through the local 
TV tuning application 601 and decodes the stream using a codec embedded 
within the media toolkit 260 (various additional encoding features of the system 
are set forth below). Once decoded, the video image is generated on the video 
display. 

^ io Figure 6d illustrates an embodiment for employing a music jukebox at a 

111 

multimedia node 150. A user initially selects a particular music play list (e.g., via 

Us? 

sj a remote control device) to be played at the multimedia node 150. The request is 

O received by a music jukebox application 601 executed on the multimedia node 

150, which forwards the request to the home media server 110. A jukebox server 
}~ 15 module 604 executed on the home media server 110 receives the request and 
forwards the request to the media server file system 606 (through the media 
toolkit 260), which retrieves the music files from the mass storage device 230. 
The jukebox server module 604 then creates the audio stream for the specified 
files and feeds the stream to the multimedia node 150. In one embodiment, the 
20 tuner server module 252 uses the RTP/RTSP protocol to create the stream (e.g., 
through communication with the RTP/RTSP module 251). The multimedia node 
150 receives the stream through its jukebox application 601 and decodes the 
TCW 21 04259.P008 



stream using a codec embedded within the media toolkit 260. Once decoded, the 
music tracks are reproduced on a set of speakers. 

Network Operations Center 
In one embodiment, a network operation center ("NOC") 180 illustrated 
in Figure 1 provides data and multimedia services for the home media server 
110. The NOC may be comprised of one or more servers communicating with 
the home media server 110 over the Internet 120 (or other network). In one 
embodiment, the NOC performs one or more of the following functions: 

Remote Monitoring and Maintenance . The NOC 180 monitors the home 
media server to ensure that it is operating within normal parameters. In one 
embodiment, the home media server 110 periodically transmits a status update 
to the NOC 180, indicating whether any particular services are required and /or 
whether any problems have arisen. If the home media server 110 fails to 
transmit a status update after a predetermined period of time, the NOC 180 may 
take steps to determine the cause of the problem and/ or take steps to contact the 
user. For example, diagnostic tests may be performed to evaluate the network 
connection between the home media server 110 and the NOC 180. If the 
diagnostic tests do not provide an answer, then the user may be contacted via 
telephone to determine the next course of action. In one embodiment the NOC 
180 is staffed by engineers or other technical assistance personnel 24-hours a day. 
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In one embodiment, the user's home security system and/or fire system 
communicates with the home media server 110 over the home media network 
140. Accordingly, the home media server 110 may be programmed to relay 
home security and /or fire system data to the NOC 180 on a periodic basis during 
5 certain periods of time (e.g., every minute). If a security problem or fire trouble 
indication is encountered during this period of time (e.g., a door/window is 
opened without the proper security access code), the NOC 180 may notify a local 
law enforcement agency and/ or the local fire department. 

q Automatic Downloading and Upgrades . In one embodiment, the NOC 

Si io 180 provides automated software downloading and upgrades to the home media 

i n 

?™ server 110. For example, when a new software interface/program (e.g., a new 

sj graphical user interface) is developed for the home media server 110 it may be 

O automatically downloaded from the NOC 180 and installed. Whether the NOC 

"SET 

H 180 should upgrade software on the home media server 110 automatically may 

S 15 be determined by each individual user. Certain users may opt to manually 

upgrade all of their software while other users may specify a particular subset of 
software to be automatically upgraded. For example, a user may specify 
automatic upgrades for graphical user interface software and manual upgrades 
for email clients and word processors. 
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In addition, the NOC 180 may store up-to-date drivers for various home 
media network devices 160-166. In one embodiment, when a user purchases a 
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new device, such as a new telephone, the device may automatically identify itself 
to the home media server 110 (e.g., via USB, IEEE 1394 or other device 
identification technology). The home media server 110 will then determine 
whether the device is supported by the system and, if so, query the NOC 180 for 
the latest driver. In one embodiment, once the driver is identified, it is 
automatically downloaded from the NOC 180 and installed on the home media 
server 110. Once installed, the driver may be updated automatically as new 
updates become available. Accordingly, driver software will no longer need to 
be bundled with home network devices 160-166, resulting an additional cost 
savings to the end-user. In addition, users will not be required to install and 
configure device drivers for each new network device they purchase. 

In addition, the home media server 110 may check the NOC 180 to 
determine whether the home network device has been certified by the NOC 180 
as meeting some minimum level of quality and/or performance. In one 
embodiment, device drivers will be downloaded and installed on the system 
only if the device has been certified. In this manner, a minimum quality of 
service may be maintained for all devices employed on the system. 

The same type of device identification, certification, and driver installation 
may be performed for communication modules 240-245 such as those described 
above. Thus, as with home network devices 160-166, certain communication 
modules 240-245 which do not meet a minimum quality of service requirement 
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(e.g., those which perform inefficient bus transactions which tie up the system 
bus 220) will not be certified by the NOC 180 and will not be permitted to be 
installed in the home media server 110. 

In addition, in one embodiment the communication modules 240-245 are 
5 manufactured and certified by third party content providers (e.g., satellite 

broadcast operators). In this embodiment, the content providers may configure 
the modules 240-245 to work with their own proprietary communication 
protocols, encryption techniques, and/ or application programs. For example, 
Q DirecTV™ may develop a proprietary interface for its users so that when a user 

*5 10 selects his DirecTV feed he will be presented with a unique user interface and/ or 
IT; channel programming environment. Accordingly, while the present invention 

\j provides a standard media transport interface for a variety of different 

O communication channels, service providers can still distinguish their services 

^ based on the proprietary applications /transports which they develop. 

Q 

15 Lo g gin g/Data Warehousing . In one embodiment, the NOC 180 may 

perform logging and data warehousing for the home media server 110. More 
specifically, the NOC 180 may maintain a log of network transactions for each 
home media server 110 and subsequently evaluate the log for a variety of reasons 
(e.g., to troubleshoot system problems, to determine a user's preferences and 

20 tailor services and/or advertising to that user, . . . etc). For example, by 

monitoring usage patterns, the NOC 180 may determine that every time a certain 
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Java applet is downloaded, the home media server 110 crashes. As such, the 
NOC 180 may takes steps to ensure that the applet in question is no longer 
downloaded by the home media server 110 (e.g., by notifying the user or 
automatically blocking the applet). The NOC 180 could then notify the technical 
support staff to determine the problem with the applet. 

Similarly, the usage log may be evaluated to determine the preferences of 
a user and to provide specialized services to that user based on those 
preferences. For example, based on the Web sites the user visits and/ or the 
channels that the user watches, the NOC 180 may determine that the user is 
interested in baseball. As such, the NOC 180 may automatically provide 
baseball-related content to the user such as, for example, broadcast schedules for 
upcoming games, a subscription offer to a sports magazine, advertisements, and 
various other baseball-related content. Similarly, the NOC 180 may determine 
that the user watches certain television shows on a regular basis, and may 
automatically download /record those shows on the home media server 110 (e.g., 
via TCP/IP), so that they will be readily available for the user (e.g., during non- 
broadcast periods of time). 

Archiving . In one embodiment, users may backup multimedia content 
and other types of data at the NOC 180. For example, a user may take a series of 
pictures with a digital camera and transmit the originals to the NOC 180 for 
developing. In one embodiment, the NOC will transmit the pictures to a 

TCW 26 04259.P008 



developer on behalf of the user and will store a backup copy of each of the 
originals (the NOC 180 will be backed up regularly to prevent loss of the 
originals). 

In one embodiment, the NOC 180 will monitor all multimedia content 
5 purchased by the user over the Internet. For example, when the user downloads 
a new compact disk ("CD") from a music download site, the NOC 180 will 
record the transaction in the user's profile. As such, the user does not need to 
store all of his/her multimedia content locally on the home media server 110. 
□ Rather, because the NOC 180 keeps track of all the content to which the user has 

^ 10 access rights, the user can offload storage to the NOC 180 and re-download the 

j] 

content when necessary (e.g., following a hard drive failure on the home media 
server 110). 



u 
n 



Re gistration /configuration . In one embodiment, users may be required to 
register with the NOC 180 upon purchasing a home media server 110 and related 

15 network devices 150-151, 160-166. The NOC 180 may prompt each user to 

respond to a series of question directed to the user's preferences, the type/ level 
of NOC 180 services desired by the user, the authorization level of each user in 
the home media server 110 household (e.g., children may be provided with 
limited limited functionality), and any other user-related data which may aid the 

20 NOC 180 in providing user services. In one embodiment, the NOC 180 will 

automatically detect the home media server 110 configuration and store this data 
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in a user database (e.g., to be used for software upgrades, troubleshooting, . . . 
etc). 

Bridge to Satellite/ Cable Operations . In one embodiment, the NOC 180 
will coordinate communication between the home media server 110 and any 
satellite/cable services to which the user has subscribed. For example, the NOC 
180 may forward pay-per-view requests transmitted from the home media server 
110 to the various cable /satellite operators, and perform the back-end processing 
(e.g., authentication, billing) required for the pay-per-view transaction. Because 
a persistent communication channel exists between the NOC 180 and the home 
media server 110 (e.g., through DSL or cable modem), no dial-up processing is 
required (unlike current pay-per-view cable and satellite systems). 

E-commerce Support . In one embodiment, rather than acting merely as a 
conduit between the home media server 110 and other e-commerce Internet sites, 
the NOC 180 may perform various e-commerce functions itself. For example, the 
NOC 180 may market and sell products on behalf of other online retailers (e.g., 
Amazon.com®). The NOC 180 of this embodiment may perform the back-end 
processing (e.g., billing and record keeping) required for each e-commerce 
transaction. 

Application Support . In one embodiment, the NOC 180 will provide 
support for applications executed on the home media server 110. For example, 
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the NOC 180 may provide a compact disk database containing CD titles, track 
information, CD serial numbers, etc. When a user copies his CDs onto the mass 
storage device 230 of the home media server 110, the home media server 110 may 
query the database (as described in greater detail below) and download title and 
track information (or other information) used to identify/index each CD and 
each track. In one embodiment, public CD databases (e.g., such as the database 
maintained at "www.cddb.com") may be filtered and improved by the NOC 180 
to remove errors. Various other types of application support may be 
implemented at the NOC 180 consistent with the underlying principles of the 
invention. 

Remote- Access Gateway . In one embodiment, the NOC 180 may be used 
as a gateway to access to the home media server 110 from a remote location. For 
example, a user from a PC connected to the Internet may log in to his home 
media server 110 through the NOC 180. 

In one embodiment, the home media server 110 is configured to make 
outgoing connections only (i.e., to reject direct incoming connections). As 
described briefly above, the home media server 110 may poll the NOC 180 
periodically (e.g., every few seconds) and transmit a status update. During this 
periodic poll the home media server 110 may query the NOC 180 to determine 
whether anyone is attempting to access the home media server 110 from a 
remote location. If so, then the NOC 180 transmits specific information related to 
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the connection attempt to the home media server 110 (e.g., authentication data 
such as user ID's, passwords, . . . etc). The home media server 110 may then 
initiate a connection with the remote user (e.g., using the remote user's IP 
address and encryption key). 

In one particular embodiment, the NOC 180 will authenticate the remote 
user before notifying the home media server 110 of the connection attempt. For 
example, upon receipt of a remote connection request, the NOC 180 may prompt 
the user to answer a series of questions (e.g., personal questions, questions 
related to the user's account, . . . etc). If the user does not answer the questions in 
a satisfactory manner, a member of the NOC staff may contact the user directly 
(e.g., via standard telephone or IP telephone). Regardless of how authentication 
takes place, once a remote user is authenticated, the user's data is transmitted to 
the home media server 110, which subsequently establishes a connection with the 
remote user. 

Additional Home Media Server Embodiments and Applications 
An external illustration of one embodiment of the home media server 110 
is shown in Figures 3 and 4 (from the back and front, respectfully). Figure 3 
shows a home media server 110 with a plurality of different media modules 320- 
325 installed in its expansion bays (i.e., electrically coupled to its system bus 220). 
As illustrated, in one embodiment, certain standard communication ports such as 
Ethernet 301, IEEE 1394 302, USB 303, digital /analog audio 305, standard 
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telephone 304, XGA/HDTV 306, and/or other standard audio/video ports (e.g., 
AV output ports 307 and 308) may be installed directly in the primary home 
media server, rather than as an expansion card. 

As illustrated in Figure 4, one embodiment of the system includes a DVD 
5 player 420 (and/ or a CD player, CD-Re Writable drive, recordable DVD drive or 
other type of portable digital media player/ recorder). The DVD player 420 is 
capable of playing a DVD directly and /or transferring multimedia content from 
the DVD to the mass storage device 230. In one embodiment, as multimedia 

i3 content is transferred to the storage device 230, either from the DVD player 420 

.a 

J3j io or over the Internet 120, an indexing module executed on the home media server 
T 5 110 indexes the multimedia content in a content database (not shown). 

s ^ Various DVD/ CD identification techniques may be used to identify the 

:^ particular DVD/CD inserted and copied to the storage device 230. For example, 

h a checksum may be calculated for a known unique portion of the DVD /CD and 

P 

15 compared with a CD/DVD checksum database (e.g., maintained at the NOC 180 
or other server). Similarly, the serial number may be read from the DVD/ CD 
and compared with a database of DVD/CD serial numbers. Additional 
DVD/CD identification techniques which may be utilized in accordance with the 
principles of the invention are disclosed in co-pending applications entitled 

20 SYSTEM AND METHOD FOR SCALING A VIDEO SIGNAL, Serial No. 

09/632,458, filed August 4, 2000; and COMPACT DISK AND DIGITAL VIDEO 
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DISK STORAGE AND PLAYBACK SYSTEM, Serial No. 



, Filed 



each of which are assigned to the assignee of the present application and which 



5 drive) is preferably large enough to store hundreds of hours of video and/or 
audio content, as well as a variety of other digital information such as telephone 
voice messages, computer programs /data . . . etc. The current recommended 
size for the storage device 230 is at least 80 gigabytes, however the particular size 
of the storage device is not pertinent to the underlying principles of the 

10 invention. 



One or more RF transmitters 430 are also provided in one embodiment of 
the home media server 110. The transmitter 430 (as well as the LAN 390, if one is 
installed) allows the home media server 110 to simultaneously transmit 



multimedia content and other types of data to the various media devices 150-151, 



15 160-166 over the home media network 140 (e.g., at least one MPEG-2 video 
stream and several audio streams). 

In one embodiment, the wireless transmission system is capable of 
handling isochronous multimedia traffic reliably (i.e., without jitter) in a wide 
range of residential settings. For example, the system must be capable of 
20 working in the presence of common residential RF interference such as 



are incorporated herein by reference. 



In one embodiment of the system, the storage device 230 (e.g., a hard 
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microwave ovens and cell phones. In one embodiment, these requirements are 
met through the proper choice of modulation and coding. More specifically, one 
embodiment employs a modified version of IEEE 802.11b adapted to operate in a 
real time environment (e.g., using Reed-Solomon forward error-correction and 
antenna diversity with circular polarization). 

Figure 5 illustrates a home media server 110 communicating 
simultaneously with a video node 520 and an audio node 522. Certain 
multimedia nodes, such as node 522, include an LCD 524 (or other type of 
display) for displaying information about the multimedia content stored on 
home media server 110 (e.g., CD and movie titles, CD tracks . . . etc). In one 
embodiment, the audio node 522 may be electrically coupled to an amplifier for 
amplifying the transmitted audio signal into a pair of speakers. As described 
above, in one embodiment, the individual multimedia nodes also include local 
storage devices (not shown) for locally caching recently used multimedia content 
and data. 

Data /commands sent from remote control devices 530, 532 are 
transmitted through the nodes 520, 522 to the home/media server 110 as 
indicated. In one embodiment, the remote control devices include an LCD (not 
shown) or other type of display for displaying information about the multimedia 
content stored on home media server 110 (i.e., instead of, or in addition to, the 
multimedia node 522 display 524). 
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As described briefly above, using the foregoing system, all of a user's 
data, music and video may be stored in a single location (i.e., home media server 
110) and accessed from anywhere in the house (e.g., stereo node 522) or the car 
(e.g., via an automotive multimedia node 164 as indicated in Figure 1). 
5 Moreover, if the home media server 110 is connected to the Internet through, for 
example, a persistent DSL connection 360, the user can access all of the stored 
content from various other locations across the globe (e.g., a summer home or a 
hotel while away on business). One embodiment of the system provides a 
secure, encrypted data stream when content /data is requested from the home 
J 10 media server 110 in this manner, thereby protecting the user's privacy as well as 

ifi the copyrights to the underlying multimedia content. 

W 

sj If a multimedia node is employed in a user's automobile as described 

O above, when the automobile is parked within range of the media server's RF 

^ transmissions, music or other audio /video content may be transmitted and 

pj 15 stored on a storage device within the automobile, even when it is not running. 

The specific type of audio /video content to be transmitted at a particular time 
during the day or week may be variable, based on users' preferences. For 
example, the home media server 110 may be configured to transmit up-to-date 
traffic reports during the morning and evening hours before the user leaves 
20 home/work. The user will then receive an instant traffic report as soon as he 
starts his car in the morning and/ or evening, followed by music or video 
selected based on the user's preferences. Various additional aspects of this 
TCW 34 04259.P008 



feature are set forth in the co-pending applications incorporated by reference 
above. 

In an embodiment which includes a CD-Re Writable ("CD-RW") drive, 
users can produce custom CDs using the audio content stored on the mass 
storage device 230 or downloaded from the Internet. The CD-RW drive may 
utilize serial copy management techniques to produce CDs which cannot 
themselves be copied, thereby protecting the copyrights to the underlying works. 
Moreover, in one embodiment, the system will determine whether the content 
owner or content creator allows copying of the multimedia content. For 
example, certain content creators /owners may allow a copy to be stored on the 
home media server mass storage device 230, but may not allow copying to a 
portable disk (e.g., a CD-RW disk). In one embodiment, content stored on the 
mass storage device 230 may be labeled as non-copyable by setting one or more 
"copy" bits to a particular value (e.g., in a bit field associated with the content). 

In one embodiment of the system, a portable music /video player node 
may be configured to retrieve multimedia content directly from the home media 
server 110 or via an adapter module. Portable MP3 players such as the Rio™ 
from Diamond Multimedia™, for example, may be used to store and play back 
digital audio content transmitted from the home media server in a flash memory 
module. In one particular embodiment, the CD-RW drive 420 is capable of 
accepting the 1/3 inch "mini CD" format. Accordingly, users may produce 
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unique mini CDs (e.g., using tracks stored on the mass storage device 230), for 
playback on MP-3 players capable of playing mini-CDs. 



MPEG conversion logic is configured on one embodiment of the home 
media server 110 (e.g., in software, hardware or any combination thereof), 
5 allowing the home media server 110 to convert from one MPEG format to 
another. Using the MPEG conversion logic, video content stored in MPEG-2 
format may be converted to MPEG-1 format and recorded on a "video CD" (a 
compact disc format used to hold full-motion MPEG-1 video). As such, users 
o may store video on a portable medium, even on a home media server 110 which 

0j 10 is not equipped with a recordable DVD drive. 

3 5*1 

Ssi ! 
'f 

Hi! 

$»& In another embodiment of the system, home appliances (e.g., the 

N 

refrigerator, the toaster, the air conditioner) and other home systems (e.g., 
^ security, air conditioning) are all provided with RF transmission devices to 

p communicate with the home media server 110. Each device may also be 

15 configured with it's own internal network address and /or Internet address. 
Users may then access information pertaining to these devices and /or control 
these devices from any room in the home or over the Internet. In one particular 
embodiment, the user's automobile is outfitted with an RF transmitter and a 
network address. Thus, in this embodiment, the automobile is capable of 
20 reporting maintenance information to the user via the home media server 110 
(e.g., low brake pads, oil change needed . . . etc). 
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In one embodiment, a user may publish a home Web page containing up- 
to-date information on each home appliance or other network device. 
Accordingly, users of this embodiment are able to monitor and control home 
appliances and systems from anywhere in the world. In one embodiment this 

5 includes the ability to select broadcast listings and direct the home media server 
110 to make recordings (e.g., based on date/ time or broadcast ID code). In 
addition, in one embodiment, users may connect remotely to the home media 
server to review email and /or voicemail listings (which may be displayed to the 
user in the form of a single, generic "message" box). Voicemail messages may be 

10 streamed to the remote user's location over the Internet or other network. 
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Telephony Embodiments 
As illustrated in Figure 7a, in one embodiment, home telephone devices 
166 may also be configured to rim through the home media server 110. In this 
embodiment, incoming faxes and voicemail are stored on the home media 
^ 15 storage device 230 and may be accessed from any room in the house (e.g. from 

telephone devices 166, personal computers 160, PDAs 165, and/ or video systems 
151/171). In addition, phone number and address information may be stored in 
a contact database 740 on the home media server 110 and accessed through the 
various telephone devices 166 (or other home media devices). Offloading user 
20 contact data from the telephone devices 166 in this manner allows telephone 
devices 166 to be manufactured will less memory and less processing power, 
further decreasing costs to the end user. 
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As illustrated in Figure 7a, in one embodiment, the home media server 
110 includes a telephone connection module 730 which coordinates between 
standard telephony calls placed /received over the public switched telephone 
network ("PSTN") 170 and calls placed over the Internet 120 using IP telephony 
5 protocols. The telephone connection module 730 will automatically route 

incoming calls from both sources to the same set of home telephone devices 166 
(or other home media devices such as the user's personal computer 160). 

Users may specify whether a particular outgoing call should be placed 
over the Internet 120 (e.g., to an IP telephony device 710) or over the PSTN 170 

10 (e.g., to a standard telephone device 720). In one embodiment, the telephone 
connection module 730 analyzes each outgoing telephone connection request to 
determine whether the call should be routed through the Internet 120 or through 
the PSTN 170. The telephone connection module 370 may factor in various types 
of connection data to make telephone connection determinations. For example, a 

15 user may specify certain contacts within the contact database 740 for which IP 
telephone connections should be used and certain contacts for which standard 
telephone connections should be used. In one embodiment, the telephone 
connection module 730 will select a particular connection based on whether it is 
the least expensive option for the user (e.g., for contacts with both IP and 

20 standard telephone capabilities). Another variable which may be factored into 
the selection process is the connection throughput available to the user over the 
Internet. More specifically, the telephone connection module 730 may initially 
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test the connection throughput (including the availability of a reliable 
connection) over the Internet 120 and place an IP telephony call only if some 
minimum level of throughout/ reliability is available. Any of the foregoing 
variables, alone or in combination, may be evaluated by the telephony 
5 connection module 730 to select and appropriate telephone connection. 

One embodiment of a protocol architecture for supporting IP telephony 
and related communication functions on the home media server 110 is illustrated 
in Figure 2b. Various telecommunication standards are supported by this 

□ embodiment, including the H.248 media gateway control standard ("MEGACO") 

'•iy 

flt 10 standard; the ITU-T H.323 and session initiation protocol ("SIP") standards for 

Ln 

»fj > multimedia videoconferencing on packet-switched networks; the Realtime 
Control Protocol ("RTCP") standard - a companion protocol to the realtime 

Q transport protocol ("RTP") (described above) used to maintain a minimum QoS; 

and the ITU T.120 standard for realtime data conferencing (sharing data among 

S : 

P 15 multiple users). 

In addition, various audio and video codecs are supported by the 
illustrated embodiment, including G.711, G.723, and G.729 (for audio); and H.261 
and H.261 (for video). As illustrated, each of these codecs may be executed 
above RTP, an IP protocol that supports realtime transmission of voice and 
20 video. Each of the foregoing IP communication protocols may be utilized by 
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media server 110 application programs through the programming interfaces of 
the media toolkit 260. 

It should be noted that, although the embodiment illustrated in Figure 7b 
includes a specific set of communication protocols, various other communication 
5 protocols may be implemented within the home media server 110 while still 
complying with the underlying principles of the invention. For example, in one 
embodiment, new protocol stacks (both proprietary and industry-standard) may 
be automatically transmitted from the NOC 180 and installed on the home media 
O server 110 as they become available. 

jp 10 Figure 7c illustrates a signal diagram representing the set up and 

f«* termination of a telephony connection according to one embodiment of the home 

media server 110. At 750, the home media server transmits a call connection 
Ji; "invitation" on behalf of User 1 to User 2. Once User 2 accepts the call, a 
□ "success" message is transmitted back to the home media server 110 at 751. In 
15 response, the home media server 110 acknowledges receipt of the "success" 
message at 752 and allocates a media stream 760 to support bi-direction audio 
communication between User 1 and User 2. To tear down the media stream 760, 
one of the users must hang up the phone, resulting in a "bye" message 753, 754 
followed by a termination acknowledgement ("success") from both sides 755, 
20 756. 
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Broadcast and Packet-Switched Channel Coordination 
In one embodiment illustrated in Figure 8b, content providers 830 may 
transmit content to home media server 110 over Internet channels 120 as well as 
standard broadcast channels 170. The transmitted content may then be cached 

5 locally in a content database 850. When a user subsequently requests the same 
content to be played back on an audio device or video device 171 (e.g., via media 
node 151), it will be replayed to the user directly from the local content database 
rather than over a broadcast channel. In one embodiment, a content selection 
module 840 will determine whether to retrieve the content through a broadcast 

10 channel or directly from the content database 850. 

In one embodiment, the home media server 110 and/ or the content 
provider 830 will monitor the preferences of each user in the media server 110 
household to determine the content which will be requested during certain times 
of the day, and/or days of the week. For example, if a user watches "the 

15 Simpsons®" at 6:00 PM, the home media server 110 and/ or content provider 830 
will record this behavior. The content provider 830 may then transmit the latest 
Simpsons episode to the home media server 110 over the Internet 120 before its 
scheduled broadcast at 6:00 PM (e.g., during periods of the day or evening when 
traffic on the Internet 120 is low). Various other pre-recorded broadcast 

20 programs may be transmitted to the home media server 110 over the Internet 
rather than over a dedicated broadcast channel, thereby freeing up a substantial 
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amount of inefficiently used broadcast network bandwidth (pre-recorded 
content currently represents the vast majority of all broadcast content). 

As indicated in Figure 8b, this embodiment of the home media server 100 
will provide a seamless mechanism for transitioning from standard digital 
broadcast channels (e.g., MPEG-2 channels) and analog broadcast channels to 
packet switched isochronous channels (for live broadcasts) and asynchronous 
channels (for pre-recorded broadcasts). Initially, the breakdown between 
broadcast and packet switched channels (e.g., TCP/IP) will be similar to that 
illustrated in column 804, with analog broadcast 803 taking up approximately 
half of the available bandwidth, with the other half split between digital 
broadcast 802 and packet switched channels 800. As content providers begin to 
switch over to a packet-switched transmission scheme, this breakdown will 
change as indicated at bandwidth allocation 805, with packet-switched channels 
taking up approximately half of the available bandwidth and the remainder split 
between analog broadcast 812 and digital broadcast 811. 

Finally, when standard analog and digital broadcast channels have been 
phased out completely, all content will be delivered over packet switched 
channels as indicated in bandwidth allocation 806. Many packet switched 
protocols support both asynchronous and isochronous data transmissions. 
Accordingly, several isochronous channels may be provided to support live, real 
time events (e.g., sports events, the evening news, . . . etc) and the remainder of 
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the bandwidth may be allocated to pre-recorded events (e.g., sitcoms, game 
shown, talk shows, . . . etc). These pre-recorded events /shown may be 
asynchronously downloaded to the home media server 110 at any time of the 
day or evening (e.g., during periods of low network activity), resulting in a much 
5 more efficient allocation of available transmission bandwidth. 

Because embodiments of the home media server 110 described above are 
capable of receiving, storing and coordinating content transmitted from both 
packet switched channels (e.g., TCP/IP) and standard broadcast channels, the 
^ transition illustrated in Figure 8b may be accomplished seamlessly, while at the 

fjj 10 same time improving the overall quality of service to the end user. 

F In one embodiment, several concurrent, real-time multimedia streams are 

Q supported by the home media server 110 through buffering and /or disk storage 

'erf 

P techniques as illustrated in Figure 8c. More specifically, incoming multimedia 

!~ content 860-862 from several different channels (e.g., standard broadcast 

Li 

15 channels 860, 861 and packet switched channels 862, 863) may be buffered in a set 
of input buffers 870-873 and output buffers 890-893 on the home media server 
110. The input buffers and output buffers 870-873 and 890-893, respectively, may 
be portions of memory allocated within the main memory 201 (see Figure 2). 
Alternatively, or in addition, the input/ output buffers may be configured on the 

20 communication modules 240-245, within the system ASIC 210, and/ or as 
separate modules on the home media server 110 motherboard. 
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In one embodiment, the mass storage device 230 reads the multimedia 
data from each of the input buffers 870-873 and writes the data to a set of 
multimedia tracks 880-883. The multimedia data is subsequently read from each 
of the multimedia tracks 880-883 to a set of output buffers 890-893, from which it 

5 is transmitted to one or more of the destination multimedia devices 150, 151, 163. 
Buffering data streams on the home media server 110 in this manner provides 
significant cost efficiencies for the entire system. Specifically, because the 
multimedia data is buffered at a single distribution point within the home media 
network 140 (i.e., the home media server 110), no buffering is required at each of 

10 the individual multimedia devices 150, 151 and 160-166, thereby significantly 
reducing the cost and complexity of these devices. 

The mass storage device 230 of this embodiment has enough bandwidth in 
its read/write operations to support several multimedia streams at once. 
Although the seek time of the mass storage device 230 (i.e., the time required to 
15 jump from one track to another) may be relatively low (i.e., relative to its 

read /write bandwidth), buffering of the multimedia data in input and output 
buffers as described ensures that the various streams will be provided to their 
respective multimedia devices 150, 151 and 160-166 without interruption. 

In one embodiment, each of the multimedia streams 860-863 operate 
20 independently of one another. For example, if playback of a particular 

multimedia stream is paused, e.g., from multimedia device 150, the home media 
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server 110 will interrupt transmission of multimedia content from the output 
buffer 893 associated with that device 150, and will begin storing subsequent, 
incoming multimedia content on the mass storage device 230 (e.g., within the 
media track 883). However, this will not affect playback at any of the other 
multimedia devices on the network 140. When playback is resumed, the content 
will be streamed from the output buffer 893 (and media track 883) from the same 
point at which is was paused (thereby providing real time, time-shifting 
functionality for live, real-time broadcasts). 

In one embodiment, an indexing module (not shown) will generate a 
content index for users of the home media server 110 which will provide users 
with a comprehensive, seamless listing of multimedia content from the Internet 
(e.g., streaming media content), broadcast channels (including live and pre- 
recorded broadcast channels), asynchronous /isochronous multimedia channels, 
and/or various other media sources/ channels. Accordingly, this embodiment 
will provide users with the ability to navigate through and select content from a 
virtually unlimited number of different multimedia sources and channels having 
any knowledge of the underlying protocols and communication infrastructure 
supporting the sources /channels. 

Intelligent Buffering and Bandwidth Allocation 
Virtually all communication channels are bandwidth-limited in some 
manner, due to the physical limitations of the underlying transmission medium 
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and /or the signaling limitations of the channel (e.g., the channel's allocated 
frequency spectrum). For example, a 100 Base-T Ethernet network is capable of 
providing a total data throughput of 100 Mbps, which is shared by all nodes 
(e.g., computers) on the network. Similarly, multimedia devices 150, 151; 160-166 
5 communicating on the home media network 140 described above all share some 
maximum network signaling rate (e.g., if a standard IEEE 802.11b network is 
employed the signaling rate is 11 Mbps). 

Because bandwidth is shared on these systems, overall network 

performance may degrade as a result of nodes performing concurrent, high- 
ly 

s f| 10 bandwidth network transactions. This can be particularly problematic on 
uj networks such as the home media network 140 which support real-time 

^ multimedia services. More specifically, high quality audio and video signals 

r S transmitted from the home media server 110 to multimedia nodes 150, 151, 

O 

require some minimum level of network bandwidth to be rendered properly (i.e., 
O 15 without distortion or interruption). Accordingly, one embodiment of the 

invention includes intelligent buffering and bandwidth allocation techniques to 
ensure smooth playback of the multimedia signal at each of the multimedia 
nodes 150, 151. 

Figure 9a illustrates a histogram of normalized bitrate data for a particular 
20 multimedia stream (e.g., a DVD) over a period of 120 seconds. Curve 902 shows 
the bitrate averaged at one second intervals whereas curve 904 shows the bitrate 
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averaged at 10 second intervals. The average bitrate over the entire 120 second 
interval is represented by curve 906 (approximately 4.7 Mbps). Accordingly, 
based on the histogram data, the effective bitrate required to properly render the 
video stream at multimedia node 150 fluctuates significantly - from a maximum 
5 throughput of 7.143 Mbps to a minimum throughput of 1.657 Mbps. 

One embodiment of the home media server 110, illustrated in Figure 9b, 
uses this type of bitrate data to allocate bandwidth and buffering levels for 
multimedia playback. Accordingly, when a user selects audio or video content 
^ (e.g., a DVD) to be transmitted to a multimedia node (e.g., video node 151), the 

m 

.m 10 home media server 110 initially retrieves a bitrate template 930 (e.g., a histogram 
yj of bitrate data) associated with the requested content from a bitrate database 935. 

^ The database 935 may be maintained locally on the home media server 110 

^ and/ or, as illustrated, may be maintained on a remote database server 940 (e.g., 

a 

m maintained at the NOC 180). In either case, the database 935 may be continually 

O 15 updated as new multimedia content becomes available. Once the bitrate 

template 930 is initially downloaded from the remote server 940, a copy may be 
stored locally on the home media server 110 for future reference. In one 
particular embodiment, the bitrate template 930 may be transmitted along with 
the underlying multimedia signal (e.g., embedded within the MPEG-2 stream 
20 read from a DVD or broadcast over a cable channel), rather than maintained in a 
separate database 935. 
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If the bitrate template 930 is maintained in a database 935, various types of 
identification data 925 may be used to identify the correct template for the 
multimedia content being played including, but not limited to, those set forth in 
co-pending application entitled SYSTEM AND METHOD FOR SCALING A 
VIDEO SIGNAL, Serial No. 09/632,458 (incorporated by reference above). For 
example, a checksum may be calculated for a known unique portion of the 
multimedia content and compared with checksums stored in the database 935. 
Similarly, if the content is stored on a CD or DVD, the CD/DVD serial number 
(or other embedded identification code) may be used to perform a database 935 



*£i 10 query. Various other identification techniques may be used to identify the 

{J? 



multimedia content while still complying with the underlying principles of the 
invention. Once the correct bitrate template 930 is identified, it is transmitted 
from the remote server 940 to the home media server 110. Alternatively, if the 
data is stored locally on the home media server 110, it is retrieved directly from 
15 the home media server's storage device 230. 

In one embodiment, an allocation module 950 running on the home media 
server 110 analyzes the bitrate template 930 to establish an efficient bandwidth 
allocation and/ or buffering policy for transmitting the multimedia stream(s). 
The allocation module 950 acts as a data "throttle," increasing or decreasing the 
20 data throughput from the home media server 110 to the multimedia nodes 150- 
151 as necessary to meet the bitrate requirements of each multimedia stream 
(e.g., through control of the home media server output buffers 910-912 and/or 
TCW 48 04259.P008 
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RF transceiver 915). The goals of one embodiment of the system are (1) to ensure 
that the underlying multimedia content is reproduced at the multimedia node 
151 without interruption/jitter; (2) to minimize the memory requirements at the 
multimedia node 151; (3) to minimize the playback delay experienced by the end 
5 user; and /or (4) to minimize the bandwidth required to accurately reproduce the 
multimedia content at the node 151. Any of these goals, alone or in combination, 
may be factored into the allocation module's 150's bandwidth/buffering 
allocation policy. 

In one embodiment, the allocation module 950 analyzes the bitrate 
10 template 930 to ensure that the amount of multimedia content in each 

multimedia node buffer - e.g., buffer 920 of multimedia node 151 - is sufficient to 
handle upcoming spikes. For example, given the bitrate template data set forth 
in Figure 9c, the multimedia node buffer 920 must have sufficient multimedia 
data (i.e., an adequate number of bits) to handle the bitrate spike between 3 
15 seconds and 4 seconds (i.e., 6.2 Mbps). As such, by analyzing the bitrate 

template 930 as a whole, the allocation module 950 may increase the allocated 
data throughput between 1 second and 3 seconds to sufficiently fill the buffer 920 
by the time the bitrate spike arrives (i.e., at 3 seconds). Thus, the number of bits 
consumed during the spike (6.2 Meg) must be less than or equal to the number of 
20 bits in the buffer at start of spike period (3 seconds) minus the bits added to 
buffer during spike period (i.e., the per-second bitrate). Otherwise, playback of 
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the multimedia stream will stall due to an underrun condition (i.e., a lack of 
multimedia data at the multimedia node 151). 

The allocation module 950 may also factor in bitrate templates 930 of other 
multimedia streams when making allocation decisions for a given stream. For 
example, when making allocation decisions for the DVD stream in Figure 9b 
(which passes through buffers 910 and 920), the allocation module 950 may 
evaluate the bitrate requirements of the other two streams (i.e., the MP3 stream 
and Cable stream passing through buffers 911, 921 and 912, 922, respectively). 
As such, if one particular stream requires a significant amount of throughput at a 
given point in time, the allocation module will take anticipatory steps to ensure 
that sufficient multimedia data will be available. For example, in Figure 9d, the 
bitrate data illustrated in timeline 960 indicates a severe bitrate spike between 40 
and 60 seconds. The spike is severe enough that in order to properly render the 
multimedia stream associated with it, the allocation module 960 may need to fill 
the multimedia node buffer by the time the spike arrives (i.e., at 40 seconds) and 
also divert a significant amount of the system bandwidth to the stream to avoid 
an underrun condition. As such, in anticipation of the spike, the allocation 
module 950 may attempt to fill all the buffers on the system including the buffers 
of the other two streams (associated with the bitrate data shown in timelines 961 
and 962) so that by the time the spike arrives, it can divert bandwidth from these 
streams to the stream with the bitrate spike. It should be noted that the 



TCW 



50 



04259.P008 



foregoing bandwidth/buffer allocation example was for the purpose of 
illustration only and should not be read to limit the scope of the invention. 

In one embodiment, the when a new playback stream is requested at a 
multimedia node 151, the allocation module 950 will initially allocate all system 
5 bandwidth available at that time to the stream. It will subsequently decrease the 
bandwidth (i.e., the "throttle") only when other multimedia nodes require 
additional bandwidth and/or when the buffer at the multimedia node 151 fills 
up. Accordingly, one goal of this embodiment is to keep the buffer at the 
y multimedia node 151 filled to its limit at all times (e.g., so that it will be ready for 

ilji 10 upcoming spikes). 

n t I 

I ™ Playback of the multimedia data from the multimedia node buffer 920 

"4 

jg may start at different times, depending on the histogram data associated with the 

□ content. For example, the bitrate for certain high definition television broadcasts 

Is. 

0 or DVDs such as the one illustrated in Figure 9e (which shows bitrate data for 

15 the movie "South Park") may start off at a relatively high value (i.e., 

approximately 8.6 Mbps). As such, it may be necessary to accumulate a certain 
amount of multimedia data in the multimedia node buffer 920 before initiating 
playback. The table 970 illustrated in Figure 9e shows the amount of data that 
must be buffered before playback (column 972), given a particular multimedia 
20 stream bitrate (column 971). The table 970 also indicates the amount of time 

which a user must wait before playback begins (column 973). As indicated in the 
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bottom row of the table 970, at a bitrate of 9Mbps, no buffering of the multimedia 
signal is required. In one embodiment, playback is initiated by the allocation 
module 950 (as indicated by the control signal 952 in Figure 9b). 

In contrast to multimedia stream described by the histogram of Figure 9e, 
the multimedia stream described by the histogram illustrated in Figure 9f (the 
DVD for the movie "Species 1") requires significantly less buffering before 
playback. More particularly, as indicated in table 975, no pre-playback buffering 
is required at a bitrate of 5 Mbps. As such, if 5Mbps or more system bandwidth 
is available, playback may begin as soon as the stream reaches the multimedia 
node 151. In addition, in one embodiment, the allocation module 950 may 
initially allocate more bandwidth than required to fill the multimedia node 
buffer 920 during playback (e.g., so that more system bandwidth may be 
allocated elsewhere once the buffer is full). 

In one embodiment, when a new playback stream is requested from a 
multimedia node 151, the allocation module 950 may initially allocate just 
enough system bandwidth to meet some minimum playback delay threshold. 
Thus, if this minimum delay threshold is set at 4.5 seconds, the allocation module 
650 must allocate at least 7 Mbps to reproduce the multimedia content 
represented by the histogram of Figure 9e (see Table 970, 5 th row). By contrast, to 
meet the minimum delay threshold for the multimedia content represented by 
the histogram of Figure 9f, the allocation module 950 may allocate a bitrate of 
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less than 5 Mbps (see Table 975). After the initial bandwidth allocation, the 
allocation module may use the histogram data to ensure that the amount of 
multimedia content in the multimedia node buffer 920 is sufficient to handle 
upcoming spikes in bitrate (as described above). 

5 Various additional bitrate histograms and related bandwidth allocation 

tables are set forth in Figures 9g through 9m. It should be noted, however, that 
these histograms are provided merely for the purpose of illustration, and should 
not be read to limit the scope of the present invention. Moreover, although the 
examples set forth above focus on the reproduction of DVD movies at 

10 multimedia nodes on a home media network, it should be noted that the 

underlying principles of the invention may be implemented on virtually any 
system in which time-based data is transmitted over a bandwidth-limited 
network. Accordingly, the scope and spirit of the present invention should be 
judged in terms of the claims set forth below. 

15 Conditional Access and Encryption 

One embodiment of a cable broadcast module for receiving one or more 
broadcast channels is illustrated in Figure 10. This embodiment includes two 
tuners 1010, 1011 for concurrently tuning to two separate broadcast channels 
(e.g., to be viewed at two separate multimedia nodes on the home media 

20 network 140). Additional tuners may be included within the cable broadcast 
module as necessary (i.e., for receiving additional broadcast channels). 
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In one embodiment of the system, a smartcard 330 (see also Figures 3) 
inserted into one or more of the media modules 320-325 may be programmed 
with a specific set of access rights defining the particular channels to which users 
are permitted access. The smartcard 330 may work in conjunction with a 

5 conditional access module 1030, which will prevent certain channels/ content 
from being transmitted on the system which the user does not have the right to 
receive. In one embodiment, different members of a family may be assigned 
smartcards 330 with different access privileges. For example, children may be 
assigned smartcards 330 which limit the channels and/or Internet sites which 

10 they are permitted to access. 

In the illustrated embodiment, the signal outputs from each of the tuners 
1010, 1011 are fed into separate audio and video analog-to-digital decoder 
modules 1024, 1026 (audio) and 1023, 1025 (video), respectively. In one 
embodiment, the signals are then re-encoded with a compression codec. For 

15 example, in the embodiment shown in Figure 9, MPEG-2 encoder modules 1031, 
1033 encode the audio/video signals into MPEG-2 format (i.e., which may 
subsequently be decoded by any multimedia devices 150, 151 on the home media 
network 140). It should be noted that various other digital compression codecs 
may be used (e.g., AC-3, MPEG-1, . . . etc) while still complying with the 

20 underlying principles of the invention. 
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In addition, in one embodiment, before being transmitted onto the system 
bus 220, the encoded multimedia signals are encrypted by an encryption module 
1060 using a unique encryption key 1061. As such, in this embodiment, all 
multimedia data stored on the home media server 110 (i.e., on the mass storage 
device 230) and transmitted across the home media network 140 are encrypted, 
thereby preventing copying and playback by any unauthorized devices listening 
in one the network 140 or on the system bus 220. Moreover, authorized 
multimedia devices 150, 151, 160-166 (i.e., those certified by the NOC 180) 
communicating on the home media network 140 are configured with the same 
unique encryption key 1061, thereby allowing them to properly decode the 
multimedia data transmitted from the home media server 110. 

In one embodiment, the encryption key 1061 is transmitted securely to the 
home media server 110 and media devices 150, 151; 160-166 through the 
conditional access system. Alternatively, or in addition, the key 1061 may be 
transmitted using conventional secure communication techniques such as Secure 
Sockets Layer ("SSL") or Data Encryption Standard ("DES"). 

One embodiment of a satellite module for connecting the home media 
server 110 over a satellite channel is illustrated in Figure 11, including multiple 
tuners 1110, 1111; forward error correction modules 1130, 1140; a smartcard 1120 
and associated conditional access module 1150; and an encryption module 960 
for encrypting content transmitted onto the system bus 220 (and across the home 
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media network 140). The different decoding, encryption, and authentication 
features described above with respect to the cable broadcast module illustrated 
in Figure 9 are also configured into one embodiment of the satellite module. 

One embodiment of a cable modem for connecting the home media server 
110 over a cable channel is illustrated in Figure 12. The cable modem includes a 
modulator unit 910, a Data Over Cable Service Interface Specification 
("DOCSIS") module 920, and a data pump /PCI interface unit 930. Various 
different types of cable modems may be coupled to the home media server 110 
while still complying with the underlying principles of the invention. 

In addition to the encryption system described above, one embodiment of 
the system provides copyright protection through the use of copyright tags 
associated with content stored on the home media server 110. The tags of this 
embodiment identify which types of content /information may be copied or 
streamed to which multimedia devices. As illustrated in Figure 13, tags may be 
set to a variety of copy /playback levels, including, for example, a tag 1311 
indicating no restrictions on copying for audio file 1310; a tag 1321 allowing only 
a single serial copy (e.g., to a CD-RW disk) with no copying to personal 
computers for audio file 1320; a tag 1331 permitting playback to local TVs (i.e., 
within the home media network 140) for video file 1330; and a tag 1341 allowing 
24-hour playback to local TVs only (e.g., similar to a pay-per-view feature) for 
video file 1340. Accordingly, this embodiment provides an additional 
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mechanism for providing secure storage and transmission of copyrightable 
material. In one particular embodiment, content providers (e.g., cable and 
satellite providers) are provided with the ability to program tags into their 
content as necessary. 

5 Embodiments of the present invention include various steps, which have 

been described above. The steps may be embodied in machine-executable 
instructions. The instructions can be used to cause a general-purpose or special- 
purpose processor, which is programmed with the instructions to perform 
?»# certain steps. Alternatively, these steps may be performed by specific hardware 

m 10 components that contain hardwired logic for performing the steps, or by any 

so 
s 

y combination of programmed computer components and custom hardware 

^ components. 

hi 

Q 

'-JJ 

O Elements of the present invention may also be provided as a computer 

3 program product which may include a machine-readable medium having stored 

15 thereon instructions which may be used to program a computer (or other 
electronic device) to perform a process. The machine-readable medium may 
include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and 
magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnet or optical 
cards, propagation media or other type of media /machine-readable medium 
20 suitable for storing electronic instructions. For example, the present invention 
may be downloaded as a computer program product, wherein the program may 



TCW 



57 



04259.P008 



• # 

be transferred from a remote computer (e.g., a server) to a requesting computer 
(e.g., a client) by way of data signals embodied in a carrier wave or other 
propagation medium via a communication link (e.g., a modem or network 
connection). 

Throughout the foregoing description, for the purposes of explanation, 
numerous specific details were set forth in order to provide a thorough 
understanding of the present system and method. It will be apparent, however, 
to one skilled in the art that the system and method may be practiced without 
some of these specific details. Accordingly, the scope and spirit of the invention 
should be judged in terms of the claims which follow. 
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